Papers with annotation method
Web-based Annotation Interface for Derivational Morphology (2022.naacl-demo)
Copied to clipboard
| Challenge: | a visual interface for manual annotation of language resources for derivational morphology is created using relatively simple programming techniques. |
| Approach: | They propose a web-based visual interface for manual annotation of language resources for derivational morphology. |
| Outcome: | The proposed interface can be used for manual annotation of derivational morphology resources. |
Chinese Lexical Substitution: Dataset and Method (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for lexical substitution (LS) are limited and limited in coverage . despite extensive research on Lexical Substitution in various languages, there is limited evidence for LS in Chinese. |
| Approach: | They propose to use human and machine collaboration to construct a Chinese LS dataset . they combine four unsupervised LS methods to generate candidate substitutes . |
| Outcome: | The proposed method outperforms existing benchmarks on the Chinese lexical substitution task. |
Optimizing Code Retrieval: High-Quality and Scalable Dataset Annotation through Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for code retrieval struggle to balance scalability and annotation quality. |
| Approach: | They propose a method that integrates functions called within the repository and information on third-party APIs to enhance the annotation context. |
| Outcome: | The proposed method improves the annotation context by incorporating functions called within the repository and information on third-party API functionalities. |
A Streamlined Method for Sourcing Discourse-level Argumentation Annotations from the Crowd (N19-1)
Copied to clipboard
| Challenge: | Existing methods for analyzing discourse-level argument annotations require expensive labor and data. |
| Approach: | They propose a method that breaks down a popular but complex discourse-level argument annotation scheme into a simple iterative procedure that can be applied even by untrained annotators. |
| Outcome: | The proposed method can be applied even by untrained annotators. |
QUD-Based Annotation of Discourse Structure and Information Structure: Tool and Evaluation (L18-1)
Copied to clipboard
| Challenge: | a new annotation scheme and discourse-analytic method is developed for information structure annotation. |
| Approach: | They propose a new annotation scheme and a discourse-analytic method based on Questions under Discussion . they introduce a tool which enables the analyst to semi-automatically segment texts and enhance them with QUDs . |
| Outcome: | The proposed method achieves good inter-annotator scores and good agreement with discourse annotations. |
Multi-resolution Annotations for Emoji Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Emojis are able to express various linguistic components, such as emotions, sentiments, events, etc. emojis have the merit of preserving information more densely, compared to words, argues a new study. |
| Approach: | They propose to use passage-level and aspect-level emoji annotations to predict the proper emmojis associated with text. |
| Outcome: | The proposed method is heuristically generated and validated with a pre-trained BERT model. |
Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing (2025.findings-acl)
Copied to clipboard
| Challenge: | Natural language interfaces are often ambiguous, vague, or underspecified, giving rise to multiple valid interpretations. |
| Approach: | They propose a modular approach that resolves ambiguity using natural language interpretations before mapping them to logical forms. |
| Outcome: | The proposed approach improves interpretation coverage and generalizes across datasets with different annotation styles, database structures, and ambiguity types. |
QA-based Event Start-Points Ordering for Clinical Temporal Relation Annotation (2024.lrec-main)
Copied to clipboard
| Challenge: | Temporal relation annotation in the clinical domain is crucial but challenging due to its workload and the medical expertise required. |
| Approach: | They propose an annotation method that integrates event start-points ordering and question-answering as the annotation format. |
| Outcome: | The proposed method achieves a 0.72 F1 score and enables collaboration among medical experts and non-experts. |
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) offer new opportunities to enhance the annotation process, particularly for detecting label errors in existing datasets. |
| Approach: | They propose to use an ensemble of large language models to flag mislabeled examples by using an LLM-as-a-judge approach to detect label errors in existing datasets. |
| Outcome: | The proposed method improves label accuracy and consistency in large language models. |